A system whose performance improves proportionally to added hardware capacity is said to be scalable. Scalability is an important property for distributed applications such as web services, because it is desirous to scale such systems as the number of users grows. In this context, developers are particularly interested in “scale out,” which means scaling a system by adding more resources as opposed to “scale up,” which means increasing the capacity of existing resources in a system.
Scalability is usually inhibited due to bottlenecks or parts of the system that are inherently slow. For example, Amdahl's law states that if “N” is the number of processors, “s” is the amount of time spent (by a serial processor) on serial parts of a program, and “p” is the amount of time spent (by a serial processor) on parts of the program that can be done in parallel, the speed up of the total system is given by: “Speedup=(s+p)/(s+p/N)”. That is, scalability is fundamentally inhibited by the serial parts/bottlenecks of the system.
A significant tension in making systems scalable is removing bottlenecks while keeping the overall system easy to use. For example, for programmers it is convenient and commonplace to use session state across several interactions. Session state refers to a set of conditions valid for a particular user session. Consider a virtual shopping chart, for instance. Here, a user adds items in sequence to the cart until the session ends with a final purchase. However, if a web service is stateful, this introduces a bottleneck into the system since each additional web server needs to access a central store where the state of all running sessions on all servers is stored. Alternatively, each server has to maintain local storage of its running sessions, which means that each session must run on the server that maintains its state thus preventing scale out.